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I. INTRODUCTION 
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(Dated: June, 2005) 

A method is introduced for studying large deviations in the context of statistical physics of disor- 
dered systems. The approach, based on an extension of the cavity method to atypical realizations of 
the quenched disorder, allows us to compute exponentially small probabilities (rate functions) over 
different classes of random graphs. It is illustrated with two combinatorial optimization problems, 
the vertex-cover and coloring problems, for which the presence of replica symmetry breaking phases 
is taken into account. Applications include the analysis of models on adaptive graph structures. 

I—* 

The algorithmic complexity of a problem is traditionally measured on an ensemble of possible inputs (instances) by 
retaining the largest time it takes for an algorithm to solve one of the instances Q. Statistical physics studies have 
however suggested a different characterization of hardness, based on the average case rather than the worst case 0. 
This alternative approach is motivated by a generic phenomenon of concentration, according to which a particular 
instance behaves almost surely as the average case in the limit of infinite size. This self-averaging property is common 
to many disordered systems whose environment is specified by a quenched random variable from a prescribed ensemble : 
in the thermodynamical limit, the properties of a sample tend to be independent of the particular realization of the 
disorder. Due to this parallel, the methods first developed for physical disordered systems have been successfully 
applied to combinatorial optimization problems 0. The interest in typical properties is however far from limited to 
physics or optimization ; information and graph theories Q are two other major fields where they play a key role, 
I ' as illustrated by the seminal works of Shannon and Erdos-Renyi respectively. 

In any practical implementation however, optimization or coding theories face large but finite system sizes. In such 
■^j- ■ situations, controlling the deviations from the typical case becomes of primarily practical interest. The scope of large 
\ deviations theory is precisely to evaluate the probability of the rare events associated with such finite size effects. 
In addition, and despite the tremendous number of their elements, large deviations theory has also a direct relevance 
for physical systems : as will be explained, it underlies the thermodynamics of systems whose configuration space is 
constituted of different realizations of the quenched disorder. Consequently, when the disorder consists of an ensemble 
of random graphs, it allows to solve models with variable topologies. A large deviations analysis is thus useful to 
address the adaptability of constrained systems, with a physical example being covalent molecular networks subject 
to stress . 

A particularly powerful method for computing typical properties of disordered systems is the celebrated replica 
I ■ method . The cavity method provides an alternative tool that yields equivalent results but has two advantages 
over the replica method : it is based on assumptions formulated explicitly, and it applies to particular instances. In 
the present paper, we develop an approach of large deviations based on the cavity method, which we call the large 
deviations cavity method (LDCM). If large deviations have only recently raised an interest in the statistical physics 
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community 0, 0, 0, ll^lblL Tl5| , they have a much longer history in probability theory , where they are notably 
used to rigorously solve statistical mechanics models 0|. At variance with this mathematical tradition, the method 
exposed here is non-rigorous and only provides a coherent heuristic framework for obtaining quantitative predictions, 
namely computing rate functions assumin g a l arge deviations principle indeed holds. Nonetheless, as for the "typical" 
cavity method of Mezard and Parisi [Til Il8( that we will recover as a particular case, the LDCM is hoped to be 
amenable to rigorous studies. 

The paper is organized as follows. The first section is devoted to introducing some basic elements of combinatorial 
optimization and large deviations theories, with an emphasis on their links with statistical mechanics. The second 
section presents the LDCM in its simplest, "replica symmetric", form : we start by rederiving in a cavity- like fashion 
Cramer's theorem, the most elementary result in large deviations theory, and then discuss different graph ensembles, 
with explicit calculations on the vertex-cover problem. The third section deals with systems having a non trivial 
internal structure : we notably generalize the large deviations approach to systems displaying a glassy "replica 
symmetry breaking" phase, a situation that we illustrate in details with the coloring problem. A conclusion closes 
the paper by suggesting some possible applications. 
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II. OPTIMIZATION PROBLEMS AND LARGE DEVIATIONS 
A. Optimization problems 

The field of combinatorial optimization provides a broad class of disordered systems, which we use here to illustrate 
the potentialities of the LDCM. We therefore start with a brief introduction to this subject (see e.g. [| for more 
details). Combinatorial optimization is primarily concerned with minimizing cost functions, £ : C i— ► R, over some 
discrete configuration space C. In view of quantifying their algorithmic complexity, optimization problems are defined 
over an ensemble X of instances /, each associated with a cost function £j . In particular, many combinatorial problems 
are defined over ensemble of graphs in which case an instance I is specified by a graph G, that is a set of TV 
nodes i — 1, . . . , TV associated with a subset of the pairs defining its edges. 

Two prototypical examples will serve as illustration. The first one is the vertex-cover problem 0], also known as 
independent set, which consists, given a graph G, in blackening as many of its nodes as possible while never blackening 
two connected nodes. The second, quite similar, example is the coloring problem |l9| which asks, given a graph G 
and q colors, whether it is possible to assign a color to each node of G so that no two adjacent nodes have same color. 
The coloring problem is strictly speaking a decision problem (the answer must be yes or no) but it is directly related 
to the optimization problem of minimizing the number of edges having two end-nodes sharing a same color : if the 
minimum is zero, the graph is colorable, otherwise it is uncolorable. 

From the statistical physics viewpoint, the cost £i[a] represents the energy of a configuration a S C, and the 
minimal cost Ei = min^ £i [a] corresponds to the ground-state energy of the disordered system having quenched 
disorder /. In this context, it is usual to introduce an inverse temperature (3 and a free energy density /j(/3) defined 
by fi((3) = — lnE CT exp(— /3£j[o])]/(J3N), such that the ground-state energy density e/ = Ej/N is given by the 
[3 — > oo limit, e/ = lim^oo fi(0). For the coloring problem, the associated finite temperature system is known as the 
antiferromagnetic Potts model [2(j , while it is called the hard-core model [2l| for vertex-cover (with {3 representing a 
chemical potential). 

For these two examples, the configuration space C is discrete : C can be taken as {0, 1}^ for vertex-cover, with 
<7j = and <Ji — 1 corresponding respectively to uncovered (white) and covered (black) nodes, and as {1, . . . ,q} N 
for coloring with cr, € {1, . . . , q} now representing the color assigned to i. Different ensembles Q of graphs, defining 
different sets of instances I, can be introduced. The cavity approach followed here applies to any ensemble of locally 
"tree-like" graphs, that is graphs whose degrees (the number of nodes to which a given node is connected) remain 
finite when TV — > oo. Three random graphs ensembles will be specifically addressed here. The first one, noted G2 , 
is the set of graphs with TV nodes where each edge as a probability 7 /TV to be present, and is known as the binomial 
model or Erdos-Renyi ensemble . The second one, noted QpJ' , and called the uniform model Q , is the set of graphs 
with TV nodes and M = 7TV/2 edges. Finally, the third one is defined through the degree distribution p(k) of the nodes 
of its graphs : each of the TV nodes has degree k with independent probability p(k) and the edges are drawn at 
random subject to that constraint. This last class notably includes random regular graphs Q, for which p(k) = S r ^ 7 
and power-law distributed graphs, for which p{k) ~ k T (with an appropriate cut-off to insure normalization). Here 

we will also consider the Poissonian model noted and defined by p(k) = 7 fc e~ 7 /fc!. In the TV — > 00 limit, the 

binomial and uniform models Q^f 1 and Gffi share with Q^f 1 the same Poisson degree distribution. This equivalence 
between the three models extends to the typical properties of optimization problems defined on them but, as will be 
shown, does not hold for atypical features. 

From the point of view of computational complexity, an important parameter is the size TV of the instances, which, 
in the case of diluted graphs, is taken as the number of nodes. As seen on the vertex-cover and coloring problems, 
the size of the configuration space C over which optimization is to be performed increases exponentially with TV, 
precluding any naive exhaustive search for large TV and possibly making the problem highly non-trivial. In fact, both 
the vertex-cover and coloring problems are known to be NP-hard in the worst case, implying that no algorithm is 
known that can solve all instances of these problems in a time growing polynomially with TV |l| . 

As stressed in the introduction, the focus on typical instances advocated by statistical physics is justified by the 
self-averaging property : when it holds for an ensemble X of instances, there is a typical value of the ground-state 
energy density e such that, for any e > 0, the probability P[\Ej/N — ej > e] for the optimum Ej to deviate from 
TVe goes to zero, P[|.Ej-/TV — e| > e] — > as TV — > 00. Informally, large problems then typically all share a common 
optimum, which, physically, can often be traced back to the equivalence of their local properties. The "typical" cavity 
method |17lll8| have been developed precisely to compute the most probable value e for problems on random graphs. 
The LDCM presented here is an extension of this approach that allows to evaluate the TV and e dependencies of 
vanishing probabilities such as P[\Ej/N — e\ > e]. 
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B. Large deviations 



For finite N, an instance has always a finite probability to deviate from the typical case. The so-called large 
deviations Q refer to the extensive deviations from Ne, of order O(N), as distinguished from the small, subextensive 
deviations from Ne, of order o(N), like for example 0{y/N) fluctuations (see however below for a relation between 
the two). The present method is based on an Ansatz, according to which large deviations are exponentially small in 
the size N of the instances, that is, the probability ¥^[Ei] for an instance I taken out of the ensemble X to have an 
optimal cost Ej is supposed to satisfy 



"N 



[E! = Ne}~e- NL(e \ (1) 



where the symbol x stands here and in the sequel for an exponential equivalence defined as limjv^oo ln(¥[Ei — 
Ne])/N = —L{e). L(e) is called a rate function, or large deviations function, and, in the simplest cases, is strictly 
positive, except for the typical value e where it achieves its zero minimum. The Ansatz is known to indeed hold in 
the solvable case where £j consists of a sum of independent identically distributed variables (Cramer's theorem, see 
Sec. IIII A"j) . and this result is robust to the presence of weak correlations among the variables (Gatner-Ellis theorem, 
to be stated below) 13. More precisely, the relation JIJ corresponds in the mathematical literature to the "large 
deviations principle" which, in its simplest form, can be stated as follows: 

Large deviations principle: The sequence {ejv}Ar e N of real valued random variables is said to satisfy the large 
deviations principle, with rate function L : R — ► R + U {oo}, if 

(i) VM>0, L(x) < M} is compact, 

(ii) for all closed subset F of M, and all open subset O of R, 

limsup-J-PA^Ar e F] < -L{F), liminf ^F N [e N eO]>-I(0). (2) 

We point out right away that counterexamples are easily found for which the previous Ansatz does not hold. In 
the field of spin-glass models, they include the two most celebrated models, the random energy model [2^ and the 
SK model [24| . For the random energy model, elementary calculations [l(J, Hl| indeed give 



P,v(e) x <( e : 6 M if£>£ " (3) 
y ' 1 if e < e, 



with s(e) = In 2 — e 2 and e = — Vfn 2. For the SK model, numerical studies a l so suggest different scalings on 
both sides of the typical value, that is Pat(e) x exp[— N a L(e)} with a ~ 1.2 when e < e, but a ~ 1.5 when e > e. 
However, in a variety of other spin- glas s models, notably including models on diluted random graphs, the Ansatz 
is supported by numerical evidence [10(. 

From the analytical viewpoint, rate functions in the context of optimization problems have been studied by Mon- 
tanari 0,Hj|, using the replica method. The results he obtained for the vertex-cover problem [l2L |25| are strictly 
equivalent to the ones to be derived here from the cavity method. Yet, as for the typical case, the cavity approach has 
the advantages over the replica method to offer a more transparent derivation, and to open the way to algorithmic 
implementations on particular systems. For a model at finite temperature 1//3, the replica method basically consists in 
inferring rate functions from the knowledge of the moments E[Z™] of the partition function Zi((3) — ex P( — P^i[ a ])i 
with E[-] referring to the average over the disorder, that is the different instances /. As far as no replica symmetry 
breaking is involved, this procedure is motivated by the following rigorous result : 

Gartner-Ellis theorem : Let {ejvjjveN be a sequence of real valued random variables and let T : R — ► R be 
defined by 

F{y) = - Hm llnE[e-*^]. (4) 

If T{y) exists, is finite and differentiable for every y £ R, then the sequence {evI^eM satisfies the large deviations 
principle with rate function L(e) given by the Legendre transform of T{y), 

-L(e)=ini[ye-F(y)}, (5) 

y&S. 

where minus signs are introduced to match usual conventions in statistical physics. We stress that for sake of simplicity, 
this theorem is stated here with much stronger hypothesis than necessary ; in particular the assumptions about the 
finiteness and differentiability of T can be relaxed pfl . 
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To apply the replica method to optimization problems, the limit j3 — > oo has to be considered, and, to obtain non 
trivial results in this limit, the replica number n must be rescaled with /3, such that (3 — > oo and n — > with y ~n/ (3 
finite. In this limit, the replica potential T(y) coincides with the function introduced in Gartner-Ellis theorem, Eq. 



e - N ^y~> = Km nZ v/P ] = ne- yE ] = / e~ N ™ + ^de. (6) 

/3^oo J 

While proceeding differently, the cavity method to be presented will lead to the same rate function, again specified as 
the Legendre transform of the potential T(y). Although both the replica and the cavity methods, based on Legendre 
transformations, naturally yield convex functions, it should be stressed that convexity is not a necessary feature of 
rate functions. In fact, non-convex rate functions are associated with phase transitions and are therefore encountered 
in many models of interest from the statistical mechanics point of view |l|| . 

Large deviations deal with exponentially small probability and may appear as only an extreme feature of finite 
size effects, while a more refined description would consist in the complete probability distribution of Ej over T. 
Interestingly, small fluctuations can be extracted from the knowledge of the rate function near its typical minimum. 
More precisely, the potential T(y) yields the cumulants of ((i?jv) fc ) c , 

((E N )% = -N^(y = 0), (7) 

where, as usual, the cumulants (X k ) c of a random variable X are defined by lnE[e* x ] = J2T=i T\(X k )c- In particular, 
the Ansatz Q predicts the variance of the small fluctuations to be generically of order V^iV, as given by the central 
limit theorem in the case of a sum of independent identically distributed variables. 



C. Statistical mechanics interpretation 



On top of their own mathematical interest, large deviations are of direct relevance to statistical mechanics studies. 
In the context of optimization problems, rate functions can indeed be interpreted as defining an entropy on the space 
of the instances X, corresponding to a thermodynamics over the quenched disorder. This relation, formalized by 
Sanov's theorem [lq . is presented here in the restricted context where X is a class of graphs associated with a given 
optimization problem. 

Viewing the ensemble of random graphs Qm as a phase space, each graph G G Qn defines a configuration to which 
is associated the ground-state energy Eq, that is, the optimal cost for the optimization problem on G. If |<7jv1 = eNs ° 
denotes the cardinality of Qn, the microcanonical entropy s(e) of the system is given by 

e N S (e) = #{G e Q N ■ Eq = Ne} = #Q N x F N [E G = Ne] x e^" 1 ^, (8) 

where #A denotes the cardinality of the set A. Thus, up to a linear transformation, the rate function L(e) is nothing 
but the microcanonical entropy s(e), 

s(e)=s -L(e). (9) 

Within this picture, the parameter y appearing in the replica method and Gartner-Ellis theorem represents the 
external inverse temperature that allows to study statistical mechanics on the configuration space spanned by the 
graphs, y = —d e L(e) = d e s(e) (y must be distinguished from the internal inverse temperature (3 which is set to infinity 
in the context of optimization). By construction, this space has no more quenched disorder, and a large deviations 
analysis appears as the statistical mechanics analysis of a pure system at finite inverse temperature y. From the 
opposite viewpoint, large deviations theory thus provides a meaning for negative temperatures, y < 0. Finally, the 
typical case is given by the infinite temperature limit, y — 0, as prescribed by replica theory. 

We stress however that the possibility of deriving the thermodynamics of the system at inverse temperature y from 
the knowledge of its microcanonical entropy s(e) is based on the equivalence between the microcanonical and canonical 
ensembles in the thermodynamical limit, which can not always be taken for granted. In presence of non-convex rate 
functions indeed, the two ensembles become inequivalent, and a first order transition occurs, whose description requires 
a Maxwell construction ; such a construction in the context of large deviations for random graph has been recently 
described in [T^| . 

We have restricted so far to the simplest case where the measure over the quenched disorder is an uniform measure 
over an ensemble of graphs, but more complicated structures can be considered as well. In particular, the disorder 
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can have different origins, as with spin-glass models |2j] or K-SAT optimization problems |l|, where in addition to the 
graph structure, the quenched disorder comprises the specification of some random couplings between the variables. 
In this case, an instance I of the problem is first defined by selecting a graph G and then by choosing the couplings J. 
Large deviations can be taken with respect to J at fixed G : for typical graphs G, the effective system still contains a 
quenched disorder (the graph) which can be handled with the usual techniques of disordered systems, but if atypical 
graphs have to be addressed as well, a second temperature needs to be introduced. The two temperatures are in 
such a case associated with two levels of probability distributions, in a construction formally identical to Parisi's 
hierarchical scheme for handling replica symmetry breaking, as will be discussed in Sec. IIV CI The same scheme also 
applies when going to lower levels to describe the internal structure of a given instance. This will be exemplified in 
Sec. IIV XI where we discuss the implications of working with a finite temperature on the instances, or working with 
optimization problems displaying a replica symmetry breaking phase. 

III. THE LARGE DEVIATIONS CAVITY METHOD 

The "typical" cavity method, as developed by Mezard and Parisi 0,0], applies to a given instance I and addresses 
the structure of its phase space, that is the organization of the configurations a € C as a function of their energy 
density £[o~]/N . The method can handle either a structure composed of a unique set (or a finite number of sets) of 
connected solutions, called a replica symmetric (RS) phase, or a structure composed of many disconnected clusters of 
configurations, called a one-step replica symmetry breaking (1RSB) phase 0. In the latter case, the crucial assumption 
is made that the number of clusters with a given energy density e is exponential in TV, 

Meters (e) ~ e N ^ . (10) 

The 1RSB cavity method is specifically designed to compute the function £(e), called the complexity, with the 
particular RS case corresponding to S = 26]. The formal analogy between the 1RSB Ansatz <|l(Jll defining the 
complexity E(e) and the large deviations Ansatz Q defining the rate function L(e) is at the root of the possibility to 
extend the typical cavity method yielding S(e) to an atypical version yielding L(e). To emphasize further the parallel, 
we introduce the function L(e) defined as L(e) = — L(e), such that L(e) plays in the LDCM a role formally identical 
to the complexity in the typical cavity method: 

F N (E = Ne)^<e NL ^. (11) 

The analogy between the complexity S(e) and the rate function L(e) should not be taken for a coincidence: the 
complexity is fundamentally nothing but a rate function [or more accurately the entropy associated to it, as in Aq. ©], 
which describes the large deviations of the energy over the different clusters of solutions. From this point of view, 
further elaborated in [23, the 1RSB cavity method is itself a large deviations method, acting on the self-generated 
(glassy) "internal disorder" of a given sample. For glassy optimization problems, being able to address such large 
deviations is crucial since ground-state clusters are atypical, that is, exponentially less numerous than clusters with 
higher energies. These atypical ground-state clusters must be obtained by correctly tuning the "internal inverse 
temperature", noted [i in this context. Remarkably, while the LDCM to be presented will also apply to the typical 
case y = 0, the 1RSB cavity method is in general not able to describe the complete complexity curve S(e), and notably 
fails to describe the most numerous, typical clusters, corresponding to /1 = 28]. 

Our presentation of the LDCM will follow closely the presentation of Mezard and Parisi of their typical cavity 
method [TtI Il8|. but major differences will show up in the way averages over the disorder are performed. To start 
with, we consider the simplest case where the underlying optimization problem is assumed to be itself RS i.e., with 
no clustering induced by its internal disorder. 

A. The cavity approach to Cramer's theorem 

Although its most interesting applications involve random graphs, the cavity method is not restricted to this 
particular geometry. As an illustration of the ideas in their simplest setting, we consider the case, with no geometry, 
of a system made of N independent elements, each contributing to the total energy £n by a random amount In 
other words, we consider here large deviations in the sum of independent identically distributed random variables. For 
such a system, the typical energy density follows from the law of large numbers, which, assuming the distribution p(X) 
of the Ai's to have a finite first moment, is e = E[A] = J xp(x)dx. Large deviations are concerned with deviations 
from the prediction £n/N = e and, for a sum of independent variables, are completely specified by Cramer's theorem, 
both a generalization of the law of large numbers and a corollary of Gartner- Ellis theorem . 
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Cramer's theorem : Let the sequence {ejv}Ar £ N of real random variables be given by ejv = (2i=i Xi)/N where 
the {Xi]i are independently identically distributed real random variables. If E[e~ aX ] is finite for all y € M, then 
{eivlweN satisfies the large deviations principle with rate function L : M — > M defined as Legendre transform of 
T : M -> R given by .F(y) = - lnE[e^ x ], that is 

-Z(e) = L(e) = inf[ye-J%)]. (12) 

2/GR 

The basic idea behind the cavity approach is to estimate the change of the system upon addition of a new variable 
(or, equivalently, upon removal of a variable, hence the name "cavity" ). Let En be the extensive energy, En = J^Li Xi. 
By virtue of the assumed independence of the Xj, the probability distribution for £ jv+i = En + A/v +1 is given by a 
convolution of those of En and Xn+i, which, with the Ansatz reads 

PiV+l(£iV+i = E) = e (Ar+1)£ (^r) = E*[Pjv(£iV = E - X)} = J p(AE)e NC ( S ^ E )dAE. (13) 

Assuming a smooth behavior of £, we write for large N, 

(N + l)C (j^rj) = NC(e) + £(e) - d t C{e) + 0(1), 

iV£ f — — j = NC(e) - AEd c C(e) + 0(1), 

where e = E/N. Setting y = d e C(e) thus yields 

T(y)=ye-C(e) = -\nE[e-y x }. (15) 
We conclude that C(e) is given by the Legendre transform of the potential T(y), 

£(e) = ey-Fiy), 

e = d y T(y), 

as prescribed by Cramer's theorem. 



B. Poissonian random graphs 

We consider now models defined on random graphs, first under the assumption that the internal structure of an 
instance is replica symmetric (RS). As a further simplification (to be relaxed later on, as for the RS hypothesis), we 
assume that the only source of quenched disorder lies in the graph structure, as it is the case for the vertex-cover 
and coloring problems. We consider here simultaneously the three ensembles of random graphs, QjJ , Q\p an d QjJ 

defined in III Al and hereafter generically referred to as Q^j'' '. In the N — > oo limit, the degrees of graphs in have 
same limiting distribution 7r 7 , where 7r 7 denotes the Poisson distribution with mean 7, ir 1 (k) — 7 fc e _7 /fc!. 

Taking a graph uniformly at random in Q N defines the measure over the quenched disorder with respect to which 
large deviations are evaluated. Following the basic principles of the cavity method, we consider the changes in the 
system when its size is increased from N to N + 1. The first idea would be to construct uniformly at random a graph 
Gn+i in Gn+i fr° m a graph Gn randomly chosen in GjJ , by connecting a new node to k nodes of Gn, with k taken 
with the distribution 7r 7 . This construction is however too naive, since, if the initial graph was in for example, 
its extension is in G~n+i> with 7'/2 = (M + k)/(N + 1) 7^ 7/2, where M = r yN/2 denotes the number of edges in 
Gn- However, for the three models, it appears that this construction yields a graph uniformly at random in <?)7+ 1 > 

with 7' = 7 + x(7> k)/N, where we have obtained for that ^(7, k) — 2k — 7. For Gn\ after addition of the new 
node the probability for an edge to be present is still y/N and not y/(N + 1), so that it is described by 7' satisfying 

y'/(N+l) =y/N, yielding x(y, k) —y. Finally for Gn , after addition of the new site, the distribution of the degrees 
7r 7 (isr) is modified to (1 — k/N)ir 7 (K) + (k/N)Ti 1 (K — 1) since k of the nodes of Gn receive an additional edge ; this 
leads to an effective distribution ir-yi(K) with 7' = 7 + k/N, so that x(k,y) = k. To sum up, we obtained 

N 11 



x(Tj k) = 2k — 7 (Uniform model G 
x(y, k) — 7 (Binomial model G^)i 
x{y,k)— k (Poissonian model G^)- 
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The fact that in all cases (x(7j k)) = 7: with the average (•) taken with respect to 7r 7 , reflects the equivalence of the 
typical properties between the three models. 

When a node is added, the ground-state energy is shifted by an amount AE. Conditioned to the fact that the new 
node is connected to k other nodes, this shift has distribution P„ (AE) from graph to graph (and from node to node 
on a given graph). Given P„ k \AE), the argument followed in Eq. (| 1 31) can essentially be repeated, 

(17) 



e (7V+l)L(B/(7V+l),7) x e Nh(e) e L(e n )-ye _ gJVL(e) g-^(j/,7) 



= £> 7 (*0 / dAEP (k) {AE)e NM(E-AE)/N^- x{l M)/N] ^ e NMe)J2^(k) / dAEP™ ( AE^* 
k=0 J k=0 

with the notations e = E/N, y = d e L(e,j), z = <9 7 L(e, 7) and !F(y,j) — ye — L(e). Eq. ijTTI) gives the Legendre 
transform of the rate function, ^(y,^), as a function of y and 2. To determine z, we need consider the energy shift 
AE due to a link addition, having distribution Pi(AE). More precisely, the average value of the energy shift when 
7-^7 + 1/N at fixed number of nodes N is required, which is obtained by adding k new edges with an appropriate 
distribution a(k). For adding a single edge results in j'/2 = (M + 1)/N = 7/2 + 1/N so we take formally 

a — <5i/2 1 where Sg(k) — 5k,e denotes the Kronecker symbol (this non-integer prescription could be avoided as in |l8( 

by adding two nodes at once instead of one). For Gjy\ we take a = ^1/2 because it corresponds to the distribution 

of the number of added edges when each of the ~ N 2 /2 edges has a probability 1/N 2 to be present in the Q^j +1 ^ N ^ 

graph, but absent in the one. Finally for G^j\ the addition of one edge leads to 7' — 7 + 2/N so that formally 
o = Si/2 as in the uniform model. A 1/N expansion of exp[7VL(e, 7 + 1/N)] then yields z = <9 7 L(e,7) as 

e z = Y<7(k) / dAEP e {AE) e - vAE ) , (18) 



k=0 



with as derived just above, 



;(7) 

5(k) = 7T!/2(fc) (Binomial model Q N 



<j(k) — 5i/ 2 (k) (Uniform model Q N } . 



a(k) — 5 1 / 2 (k) (Poissonian model Gjf )• 



As in the typical cavity method 0,0], the distributions P„(AE) and Pg(AE) can be calculated by means of 
cavity fields. The fundamental assumption made at this stage is that the nodes to which a new node is added are 
independent in the absence of the added node. Under this assumption, the problem on a random graph is reduced to 
a problem on a tree with self-consistent boundary conditions (the so-called Bethe lattice) . While the same procedure 
applies to other optimization problems, we restrict here for simplicity to the vertex-cover problem, for which we take 
the ground-state energy as the minimum of non-covered nodes under the constraint that neighboring nodes cannot be 
both covered. A recursion is written for rooted-trees with same degree distribution 7r 7 (fc) as the graphs (see Fig.lT|l. 
In general if the degree distribution is p(k), the root must be assigned the distribution q(k) = (k + l)p(k + l)/(k), 
which corresponds to the probability, when the edge (i — * 0) is chosen at random, that i has k neighbors in addition to 
; the Poisson distribution has however the specificity that q(k) = p(k) = n 1 (k). For the rooted tree with root-node 

i, let Eq 1 ^ ^ be the minimal energy with the root constrained to be non-covered (white), and e[ 1 ^ ^ the minimal 
energy with the root constrained to be covered (black). These quantities are related to those associated with the k 
rooted trees generated by deletion of the edges originating from i (see Fig. ^1 by 

k 



3= 

E ( r 0) = ][>;r 4) . 

i=i 

A cavity field is defined for each oriented edge as ft/ w0 ) = Eq^ ^ — min^g ~ *° , E^^ -') ; it satisfies the relation 



fc(i-o) = = max I 0,1 - J^ftW-O I . (20) 

j'=i 
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FIG. 1: Rooted trees in the vertex-cover problem. This figure illustrates the recursion expressed by Eq. I|19[l : on the left tree 
the root i (in absence of 0) is constraint to be uncovered, while on the right tree it is constraint to be covered. The coloring in 
gray indicates that the nodes are neither constraint to be uncovered, nor to be covered. 



The addition of the new node i is associated with an energy shift given by 



3=1 

(21) 

AE^({h^^}) = min [ l.J^/i " 
We will also need the energy shift corresponding to an edge addition, which reads 

AE cdgc = min(E^ + E^ , E^ + ^ , ^ + ^ ) - rmn(E^ , E^ 2) ) min(i#^ , E [ 2 ^ ) 
= AE e (h^ 2 \h^) = mm(h^ 2 Kh (2 ^). 

(22) 

With the help of these equations, the distributions P^{AE) and Pi(AE) are easily obtained once the distribution 
for the fields P(h) is known. 

Again similarly to the typical cavity method [ItI llg| . to derive the equation satisfied by P(h), we introduce 
R^ +1 (h,E), the probability to get an energy E and cavity field h when taking at random a graph in G^j\i an d 
choosing one of its node as a root. By definition, the marginalization over h gives F$ +1 (E/(N + 1)), the probability 
to get a graph in SfJ+i with energy E, 

J dhR% ] +1 (h, E) = e (JV+DL(s/(^+i),7). (23) 
Generalizing slightly Eq. 117(1 . we write 

R%l 1 (h,E) = jr^(k) J dAEQ^(h,AE)e NL ^ E - AE ^ N ^-^^/ N \ 

fe= ° oo , (24) 

xe^'^^W / dAEQ^(h,AE)e- yAE ~ z ^ k \ 

k=0 

where Q^(h,AE) denotes the joint distribution of the cavity field h and the energy shift AE. As in the typical 
cavity method, we verify that h is independent of E, more precisely, 

R%L(h,E) = e NL ^e-^ y ^P(h), 



1 oo _ k 

fc= •* i=l 

oc „ 

Z = ^7r 7 (fc) / dAEPW(AE)e- vAE e-* x ^' k \ 

k=0 J 

f(y, 7) = 2/e - L(e, 7) = - InZ, 



(25) 
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where P{h) also depends on 7 and y. In the particular case where x(7j does not depend on fc, the relation for 
P(/i) formally corresponds to what is known in the literature as a 1RSB factorized equation with 1RSB parameter y 

PHHl]; this is the case with Q^p but not with and 0$ . 

Specializing now to the vertex-cover problem, the equations are simplified by the fact that h S {0, 1}, so that the 
distribution P(h) can be parameterized by a single real 77, with P(h) = r)S(h— 1) + (1 — j])6(h). As seen from Eqs. (fT7|) 

and iJTSJl, the distributions Pn\AE) and Pi(AE) are needed only through their Laplace transforms, which are given 
by 

k 

J dAEP^(AE)e-y AE = J fldhtPih^e-v^^}) = e -v + (l-e-y)(l- V ) k , 



2 

J dAEPi (AE) e~ yAE = J f[dh l P{h t ) e - yAE ^ hlM) = 1 + (e^ - l) v 2 . 



(26) 



As a first check, one verifies that for y = 0, the typical RS ground-state energy density e- 1 ' is reobtained |30(, with 



same value for the three ensembles Cjffl , and G^ , 



£ (7) = 1 - T] - 7^72, 



r) = e - ™. 

The equations for y ^ can also be written explicitly. For the ensemble G^ , they read 



(27) 



1 + e-y{e^ - 1) 2 
F(y, 7) = - \n[e- y + (1 - e~ y )e-' iri ] + ^e - " - l)f] 2 - 



(28) 



For the ensemble GpJ , 



1 + e a (e^ 6 - 1) (29) 
•^(y, 7) = - ln[e" a + (1 - e- y )e-^ e ^] + 7 (1 - e~ z ) - 7 z/2. 



And for the ensemble G^\ 



*? = 7T Z77 * - TV * = iMl + (e-"-iy], , 
1 + e y(e^ Tle —1) 2 C3Q-) 

7) = - ln[e-' tf + (1 - e^e -7 " 6 ""] + 7 (1 - e" 2 ). 

The formulae l|29[l coincide with the result of the replica computation presented in • The three corresponding rate 
functions are plotted in Fig. [21 for 7 = 2. 

A remarkable aspect of the vertex-cover problem is the presence, in the typical phase diagram, of a continuous 
phase transition at 7 C = e ~ 2.71, from an RS phase at 7 < 7 C to a presumably full-RSB phase at 7 > j c .31]. Due 
to its continuous character, the phase transition can be located by analyzing the stability analysis of the RS Ansatz. 
Extending the stability analysis from typical to atypical graphs thus provides, in the ("Y,y) plane, a phase diagram 
with an "AT line" Q separating a RS phase from a full-RSB one. The three ensembles are not equivalent with respect 
to properties associated with atypical graphs, and we concentrate here on the binomial ensemble Gjf 1 ■ 

RSB effects are much likely to appear first for negative values of y, corresponding to the most frustrated graphs. 
The failure of the RS approach can in fact be inferred from an asymptotic analysis of the y — > — 00 limit : it yields 
rj ^ e y / 2 /y/j, e(y = —00) = 1/2 and L(y — —00) = (1 — ln7)/2. Clearly, this is inconsistent as soon as 7 > e since 
then it predicts then L(e = 1/2) < 0. The value thus obtained coincides with the value of j c for the failure of the 
RS approach to typical graphs [30II32I (the reason for this correspondence is elucidated below). The negativeness of 
the rate function is however a sufficient but not necessary sign of RSB. A more refined way to detect it consists in 
studying the stability of the RS large deviations Ansatz. For the binomial model, it happens to be strictly equivalent 
to the stability analysis of a factorized 1RSB Ansatz |33j, and reads 

(-i-nfe-y < 1. (31) 
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FIG. 2: Rate functions for the optimal energy e in the vertex cover problem with the binomial model Q^p (label B), the 
Poissonian model (label P) and the uniform model GpJ (label U), all three for 7 = 2. The common minimum at e ~ 0.39 
corresponds to the prediction of the typical RS cavity method (y = 0) [30| . The larger curvature of the rate function for the 
uniform model with respect to the two other models can be interpreted by the fact that fixing the ratio of edges imposes more 
constraints on the graph than fixing the degree distribution or edge probability. 
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FIG. 3: Phase diagram of the vertex-cover problem in the 
(7, y) plane for the Erdos-Renyi ensemble . The full 
line corresponds to the line 1/0(7) where the RS solution 
becomes instable ; there is no instability below the perco- 
lation threshold, 7 < 1 [2/0(7) — * —00 for 7 — > f + ]. The 
line y = reproduces the phase diagram of the typical 
case with 7 C = e ~ 2.71 being defined as the intersection 
of 2/0(7) with y = 0. 



FIG. 4: Phase diagram of the vertex-cover problem in the 
(7, e) plane for the Erdos-Renyi ensemble Q^p ■ The full 
line represents the typical RS energy density, as given by 
Eq. 12711 . and the thin line, starting only from 7=1, the 
energy density above which the RS approach fails, as given 
by Eq. (|3"T). The two curves cross at 7 C = e, corresponding 
to the onset of RSB on typical graphs. 



It starts to be violated at y = — 00 for 7 > 1, while the typical graphs described with y = are not concerned before 
7 = e. Indeed for 7 < 1, the RS Ansatz is stable for all y: (7??) is a decreasing function of y and for y — > — 00 
the asymptotic analysis yields (jr])' 2 e~ v ~ 7. At 7 = 1, only the y = —00 point, corresponding to the maximum 
achievable energy e = 1/2, is marginally unstable. Finally, for 7 > 1, there is a critical y c such that RS is stable for 
y > y c but unstable for y < y c ; y c increases when 7 increases and reaches y c = for "f c = e, the point where the 
typical problem undergoes the RSB transition. For 7 > 7 C while typical graphs are FRSB, some less frustrated graphs 
are still RS. The resulting phase diagram is shown in Fig. [3] in the plane (j,y) and in Fig. 0] in the plane (7, e). The 
occurrence of RSB at 7 = 1 is particularly interesting because this point corresponds to the percolation threshold of a 
giant connected component Q, which appeared totally irrelevant when restricting to typical graphs |3fl |. In contrast, 
when atypical graphs are included into the picture, the emergence of a giant component seems to be responsible for 
the onset of RSB, as shown in Fig. [3] (a similar analysis of the uniform model however reveals that in this case RSB 
appears only above an average connectivity 7 = 2). 

The opposite y — ► +00 limit is also interesting since it is always correctly described by the RS Ansatz, with 
e(y = 00,7) = and C(y = 00,7) = —7/2. It can be interpreted as corresponding to graphs with no edge at all, 
which occurs with probability 

P^non frustrated) x P^(no edge) X (l - -1)* x e~ Nl/2 . (32) 

Similar relations between the y — 00 limit and the probability for non-frustrated samples have been reported in a 
variety of other models [ll| , providing consistent checks of the method. 
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C. Random graphs with given degree distributions 



The LDCM applies as well to graph ensembles with non-poissonian degrees, and an example is provided here with 
graph ensembles specified by their degree distribution p(k), that is with each node having an independent probability 
p(k) of being of degree k. The reasoning for arbitrary p(k) basically follows the procedure used for the Poissonian 
model Q^f 1 which was a particularly case where p(k) = 7 fe e 1 /k\. 

We thus first consider how the ensemble is modified when a new node is connected to k nodes of a graph made of 
N nodes having degree distribution p(K). The degree distribution becomes p'(K), with 



p'(K) 



1 



k 
N 



p(K) + -p(K-l), 



(33) 



since a given node has a probability k/N to get its degree increased by one unit (the probability that it is increased 
by more than one unit is 0{1/N 2 ) and is therefore neglected). Writing p{K) = ^2 r Pr^K,r, we explicitly have 
p' r = Pr + (k/N)8p r with Sp r = p r +i — Pr- The set {p r } can serve as a characterization of the graph ensemble, and 
following the same scheme as for Poissonian graphs, we obtain 



oo „ 

Y,Pk / dAEPi k \AE)e~ yAE e- kz 

k=0 J 



with as before y = <9 e L(e, {p r }) and now 



gLfc {Pr}) 
dp r 



To get z, we notice that Eq. 1)33|) with k — 2 describes the effect of an edge addition so that 

e 2z = J dAEP e (AE)e- yAE . 
The potential whose Legendre transform yields the rate function can therefore be written 



In 



f^Pk I dAEPW(AE)e- yAE ( f dAE' P e (AE') e - yAE ' 
.fe=0 ^ 



-fe/2' 



Similarly, the cavity equation for the fields reads 



vtu \ 1 i k + l )Pk+i 
P(h Q ) = 7F 2^ 



k=0 



(k) 



Y\dh 3 P(h 3 )5{h ~ U k H{h }))e- yAE ^^ h ^e- kz 

3=1 



(34) 



(35) 



(36) 



(37) 



(38) 



where Z is the appropriate normalization and the presence of (k + l)pk+i/ (k) instead of pk reflects the fact that an 
oriented edge is chosen at random, and not a node as in Eq. I|37|) (only for Poissonian graphs do these two probabilities 
happen to be the same). 

A case of particular interest is when p(k) = §h,ri corresponding to random r-regular graphs. In such a case, the 
factor e~ kz in Eq. I|38|l is a constant which can be absorbed into the normalization Z' = Ze( r-1 ) z , 



P{h ) = w / Hdh.Pih^iho-h^dh^e-y^ 

J 3=1 



(r-l) 



({hi}) 



(39) 



This is formally identical to what is known as a 1RSB factorized cavity equation |18ll29fl . The correspondence extends 
to the formula for the potential, Eq. (|37|l . which becomes for regular graphs, 



Hv) = - in 



J dAEP^(AE)e- yAE 



2 



dAEP e (AE)e 



-yAE 



(40) 



As a consequence, the 1RSB complexity of models defined on random regular graphs coincides with minus a rate 
function, as already noticed in [Tl| . Obviously, the correspondence holds only within the RS approximation that has 
been assumed throughout since by nature of a 1RSB glassy phase, the complexity is positive while a rate function is 
necessarily non-negative ; it will be shown below how the formalism needs to be extended to include the possibility of 
RSB. When the factorization does not hold, the correspondence between rate functions over the external disorder and 
negative complexities is only approximate ; we have seen however that for vertex-cover on CJ^' the rate function starts 
getting negative values precisely at the point 7 = e where typical graphs undergo a RSB transition, in agreement with 
the observation that the two quantities are approximatively related. 
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IV. MULTI-STEP LARGE DEVIATIONS 



The LDCM can naturally be extended beyond the simple case of zero-temperature systems in an RS phase with 
disorder only specified by a random graph ensemble. We consider here successively finite-temperature systems, models 
with RSB phases, and external disorders including random couplings. In the three cases, a second temperature is 
needed to describe the large deviations with respect to the additional source of randomness. In each case also, the 
equations display a common 2RSB-like structure @j which would be promoted to the nRSB type with n > 2 if n 
different sources of disorder were present. 



A. Finite temperature 



The simplest extension requiring multi-step large deviations consists in generalizing the description of a model on a 
given graph from zero to finite temperature. Two inverse temperatures are now required : /3, for the thermodynamics 
on a given graph, and y for the large deviations in the graph ensemble. More precisely, large deviations now concern 
the density of free energy f(/3), with the limit (3 — > oo giving back to the large deviations for the ground-state density 
energy e = lim | s_ ) . o /(/?), as discussed so far. For any fixed value of /3, the rate function L(/,/3) = —L(f,(3) is 
calculated as before through the Legendre transform of a potential J-(y, (3) satisfying 

e -N W) = f df e Niu m - vf ] = i z G ((3yw, 

GeG (41) 

Z G ((3) ee e ~ 0N ^ ee £ e^ c \ 
cec G 

where we introduced Zq(J3) the partition function on the graph G at temperature /?, Cq the set of configurations on 
the graph G and #Q the cardinality of the ensemble of graphs Q. Note that the particular choice y = (3 corresponds 
to the uniform measure over all configurations {C £ Ca}GeG '■ 

J2 e- mC) = J2 e~ 0NfGW = (#G)e~ 0Nnm - (42) 
GegceCc Geg 

From the technical point of view, we just have to replace in all formulae the functions ({hi}), AE^\{hi}) 

and AEi(hi,h2) by their finite-temperature extensions hP^ ({hi}; 0), AFn ({hi}; (3) and AFi(h%, h%; f3). Taking the 
vertex-cover problem as an example, these quantities are derived by writing recursive equations for the conditional 
partition functions Zq^°\/3) and Z^~*°\0) instead of the conditional ground-state energies E^ 0) and£^ 0) . More 
precisely, Eqs. (|19|1 are replaced with 



Z { r°\f3)=l[z^\(3). 



To get lim^oo = h^^ with h^°*> defined in Eq. the cavity fields at finite temperature h^°\(3) 

are defined as 



^(P) - ~ ^ "° i?' • (44) 




With these definitions, the different functions required to compute the rate function L(f,(3) are 

h^({h j },f3) = ^{l + e^U^)), 

AFi k \{h 3 };(3) = -lln (e^ + e~^U ^) , (45) 
AF i (h 1 ,h 2 ;P) = ~ln (e~ 0hl + e- 0h > - e- ( hl+h >^ , 
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which all reduce as it should to ({hj}), AE^({hj}) and AE e (h 1 ,h 2 ) given in Eq. and (J23) when 

(3 — > oo . The only practical difference with the f) = oo case is that the distribution P(h) has no more reason to be 
peaked on integers and therefore cannot be parameterized by a single real number. 



B. Replica symmetry breaking 

The phase space of a glassy 1RSB instance is structured into clusters whose energy is controlled by a parameter /i 
in exactly the same way the finite inverse temperature (3 controls the equilibrium configurations according to their 
energy. Therefore, the extension of the LDCM from paramagnetic (RS) systems to glassy (1RSB) systems, is formally 
similar to the extension from zero temperature to finite temperature. The counterpart of Eq. I|41fl reads 



# G a&g 



^ e^ N ^ = f de 



(46) 



where 4>g(p) is the 1RSB potential on graph G and L{cf>, /i) = — L(0, /i) is the rate function describing the large 
deviations of <fi(fi) over the ensemble of random graphs ; in these formulae, we reserve greek letters for quantities 
related to the internal structure and use a to index the clusters. The saddle points in Eq. (|46ll lead to the following 
Legendre transform relations : 

F(y,v) =#-L(0,/i), y = d^L(<j>,fj,), (47) 
^(fi) =/xe-£(e), fi = d^(e). (48) 

(49) 

These quantities are computed by applying the standard 1RSB cavity method ^3] to a given set of atypical graphs 
characterized by their ground-state energy density eo- If pAr(e|eo) denotes the distribution of their clusters, the 
corresponding complexity is defined as pjv(e|eo) x exp[./V£(e|eo)] ; for an energy e close to the ground-state reference 
energy eo, it becomes 

p N (e\e Q ) x e^ N ^ (50) 

where fi = <9 e £(e = eo|eo) defines the 1RSB internal inverse temperature. The shift in energy AE induced by a node 
addition, which is needed in the recursion at the level of the graph average, is given by the shift in the reference 
energy, that is, 

PN+i{e\e Q ) =p N (e\e )e-^ AE . (51) 

The expression for the reweighting term 5 = e -^ AE i§ read from the 1RSB cavity recursion which involves n(/i), the 
distribution of cavity fields over the clusters, and P[H], the distribution of the ETs over the graphs ; for a given (class 
of) graph, the connection of a new node to k other ones indeed yields 



(52) 



n = n«[{iT}], with P[{n i }]w4/nn i (*4( i )(W))e-' AL: '' l(|/ ' |; 

k 

5 = e-» AE = S«[{IT}] = f f[ll % (h % )e-^ AE ^^. 

J i=i 

Therefore, at the level of the graph average, we have for Poissonian graphs 

-. oo k 

Ppo] = ~$> 7 (*0 / JJ ©HiPpi] s[ii -nW[{nj] s«({n,})]^ e ~ z ^\ (53) 

fe = 8=1 

and for graphs with fixed degree distribution 

p t n o] = 4 £ {k+ lk) k+1 I Il PII « p M ^ - n (fc) ({ni})] s^m}] v ^ e~ kz . (54) 
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The 1RSB large deviations equations have thus the structure of a typical factorized 2RSB theory, as the RS large 
deviations equations resembled a typical factorized 1RSB theory. In particular, for y = 0, the non-factorized 1RSB 
formalism is exactly recovered. 

Replica symmetry breaking (RSB) is relevant to many optimization problems, and the vertex-cover problem already 
provided us with such an example. For this problem, studying the local stability of the RS Ansatz was enough to 
locate the continuous transition to a RSB phase. However, other problems may display a different kind of glass 
transition, known as a discontinuous 1RSB transition, which, due to its discontinuous character, can only be correct! 



tly 

4], 



described by implementing a 1RSB formalism. Such a transition is found for instance in the coloring problem 3 
which we take here as an illustrative example of the broader class of constraint satisfaction problems. 

In the statistical physics point of view, a problem is satisfiable (SAT) if it has a zero ground-state energy density, 
e = 35] . In presence of a clustered glassy phase however, an alternative characterization is provided by the sign 
of the complexity So, giving, when positive, the number exp[iVEo] of clusters with e = 0. This complexity So is 
obtained in the 1RSB formalism by taking the limit /i — ► oo ; for the 3-coloring problem on Erdos-Renyi graphs Gjj\ 
So is found to be positive only in a restricted interval [7d,7c], with 7^ = 4.42 and j c = 4.69 j^S], as schematically 
represented in Fig. The threshold values 7^ and j c , which also appear in other constraint satisfaction problems 
such as K-SAT [3(J, locate two phase transitions, called respectively the clustering and SAT-UNSAT transitions : 
with probability one when N — > 00, a graph with 7 < 7^ is colorable and RS, a graph with 7^ < 7 < j c is again 
colorable but RSB, and a graph with 7 > "f c is uncolorable. 

As an illustration, we consider here the 3-coloring problem on the Erdos-Renyi ensemble GjJ • Following [37j . 
Eq. (53), the shift Acf>g in the 1RSB potential due to a link addition is given by 

e -Mfc = 1 + ( e -M _ \) qnm , (55) 

where the ry/s, with distribution p{rf), represent the 1RSB cavity fields for this problem [3?]]. In the LDCM, we need 
the Laplace transform of the distribution Pi(A<j>), which thus reads 

[ dA^A^e-y^ = / II d Wfo) i 1 + - Vwm)^ ■ (56) 

■* •* i=l,2 

If only SAT configurations are to be addressed, the general 1RSB-LDCM equations can be simplified by taking the 
limit [i — > 00. This limit enforces e — > and fJ,<f>{n) — > —So and requires to rescale y by taking y — ■» 00 with x = y/fi 
fixed, such that !F{y, /x) — * F{x) with 

F{x) = xEo -L(S ), x — <9 So L(S ), (57) 
where L(So) = lim M ^ oo L(0 = — So//i,/x). In this limit, 

f dA<f>P e (A<f>)e-y A * - f J[ dtfcpfo) (1 - qm^T ■ (58) 

■* •* <=1,2 

Similarly for site addition, we have, again in the limit fi — > 00, 

k 

I ' dA4>P^(A4>)e-y^ - fndwfe)^)^,...,^) 1 , (59) 

i=l 

with 

q-l , v k 

where A(f>n refers to the shift in potential due to the connection of a new nodes to k old ones (see Eq. (56) in [37|). 
The distribution p(rj) is determined, in the limit fi — > 00, by the self-consistent equation 

-.00 p k 

pM = -$> 7 (fc) / Hdwto) 6( m -f,^({r H })) aW^})* e-**<*">, (61) 

fe=0 i=l 



S« - Hm e"^ = $>tf ( t l 1 II (1 - {I + Ufc) - (60) 



with 



>7 (fe) ({%})) = ATITT? ^ E(~!) £ /) 0- ~ ( £ + V*) ■ ( 62 ) 
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FIG. 5: Rate functions L(Eo) of the complexity Eo in the 3-coloring problem for the ensemble with 7 = 4.3 < ja, 
7 = 4.45 6 [7d,7e] and 7 = 5 > j c , as obtained from the 1RSB-LDCM. A maximal value Xd of the slope x — —ds a L(Y,o), 
associated with a maximal value of Eo, is found above which a non-trivial L(Eo) ceases to exist with, by definition of 7^, 
Xd("f) < when 7 < jd, and Xd(l) > when 7 > jd. Note that for 7 = 4.3, the curve displayed certainly does not describe 
the actual rate function which is expected to vanish at Eo = : since the non-trivial solution shown coexists with the trivial 
solution reduced to the point (Eo =0, L = 0), the correct rate function may be obtained by a Maxwell construction i.e., by 
drawing the supporting line originating from (Eo = 0, L = 0) and tangent to the curve. 




FIG. 6: Pictorial representation of the phase diagram for 
typical instances of the 3-coloring problem on Erdos-Renyi 
graphs (jjy 7 ' . For 7 < jd the set of solutions forms a single 
connected cluster. For jd < 7 < 7c, the set of solutions 
is organized into exp(iVEo) clusters, where Eo is the com- 
plexity curve represented in the upper part. For 7 > 7 C , 
Eo < 0, indicating that there is (typically) no solution, i.e., 
the system is UNSAT. 



FIG. 7: Phase diagram of the 3-coloring problem on Erdos- 
Renyi graphs in the (7, x) plane. The top curve gives £d(7), 
the value of x at which a 1RSB solution appears and the 
bottom curve gives £ c (7), the value of x at which the com- 
plexity vanishes. The intersection of these two curves with 
the line x = gives back the typical phase diagram of 
Fig. HJwith the intersection points 7^ and 7 C . (Note that 
this diagram includes values from non-physical branches as 
discussed in Fig.|S]for 7 = 4.3.) 



This equation can be solved numerically using a population dynamics algorithm |ljj : as for the typical case |37j . 
recovered here by taking x — 0, a peak at rj — is observed, so that p(rj) can be written p(rf) = t6(i]) + (1 — t)p(rj) 
where p represents a continuous part. Rate functions L(Eo) obtained with this procedure are presented in Fig. [5] 

Interestingly, for any value of 7 clustering and SAT-UNSAT transitions are found to occur within atypical graphs. 
These phase transitions are found by monitoring the parameter x, which, roughly speaking, characterizes the degree 
of frustration, with larger x corresponding to less frustrated graphs. For a given 7, we indeed find thresholds x c (7) 
and Xd("f), with 2^(7) < 2^(7), such that for x < 2^(7) the graphs are UNSAT (Eo < 0) while for x > 2^(7) no more 
clustered solution is found (So = 0). The global phase diagram in the (7, x) plane is presented in Fig. [7| The typical 
phase diagram of Fig. can be read on the line x = 0, with the thresholds 7 C and 7^ determined respectively by 
x c {lc) — and Xd(jd) — 0. We also expect that, for some values of x, the 1RSB Ansatz does not hold, in analogy to 
what is found in the typical case |38| ; we however do not discuss this issue here, which could be handled by extending 
the stability analysis performed for typical instances |33|. 

In cases such as coloring on regular graphs where the cavity equations are factorized, an interpretation of J-(y, p.) 
can be given in terms of the probability Pat(£ = 0) for a graph to be SAT while lying in a class of typically UNSAT 
graphs (or conversely for a graph in a typically SAT ensemble to be UNSAT). For a RS system first, it has been shown 
in IIII CI that the rate function L(e) is given by the negative 1RSB complexity S(e), L(e) — — E(e) ; in particular, the 
probability to be SAT for a graph in the typically UNSAT phase is P^e = 0) x e NT, °, where as before S = S(e = 0). 
In the typically UNSAT phase of a 1RSB system one still has Eo < 0, but the fluctuations of the complexity Eo from 
graph to graph described by the rate function L(Eq) have to be taken into account. The relation with Pjv(e = 0) thus 
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becomes 

¥ N (e = 0) X / dS e - NL ™e N *° (63) 

J ~ oo 

since e NS ° must now be multiplied by the probability e~ NLt ^ T ' ^ to actually get a complexity E . Two cases can 
then arise : the saddle point can lie at the boundary T,q — 0, in which case P/v(e = 0) x e NL ^ s °~ ' , or can 
strictly negative, So < 0, in which case Pn{^ — 0) x e -JV[L(£o)-£o] g Q given by 1 = ds L(T,o). An alternative 
formulation can be given with J^(x) related to £(£ ) through 

e -NH*) = J e -AT[£(=o)-»E a ] _ (64) 

Since S = d x !F{x), one has to compute the maximizing J-(x), which is also associated to the saddle point 
x* = &£ L(Eo = 0) : x* < 1 corresponds to the first case with Pjv(e = 0) X e -Nf(x ) ^ w j 1 j^ e > i corresponds to 
the second case with Pjv(e = 0) x e"^^ 1 "' (the very same mechanism underlies the selection of the 1RSB parameter 
in the typical cavity method at finite temperature |17|). Physically, this second situation, x* > 1, refers to very rare 
graphs with extremely small frustration, which are thus in a RS phase [it can be seen that Tix = 1) indeed gives back 
the RS rate function L(e = 0)]. In the phase diagram of a problem like coloring, such a situation is expected only for 
the largest values of 7. 

As such, the interpretation applies only for models where the factorization holds. Otherwise negative complexities 
S(e) < should be interpreted as giving the probability e NS ^ for a model with typical complexity to have a cluster 
with energy e, which, because of the interference between the internal disorder and the local external disorder, is 
not associated with a rate function relative to the external disorder only. The estimation of Pjv(e = 0) along the 
lines presented above is however still amenable, but one has to perform a two-step large deviations analysis involving 
a second rate function L(Lq) over the first rate function Lq estimated under the RS assumption ; technically, this 
computation is quite similar to what has been done here for L(So), which can be taken as a factorized approximation 
of L(L ). 



C. Multiple sources of disorder 



In problems other that vertex-cover or coloring, the definition of an instance can include some quenched values of the 
coupling constants, as happens for spin-glass models or if-SAT. In this case, the energy shifts include a dependence 
on the couplings J, with functions AE^n'^ Jl ^\{hi}) and A£^ Jl2 ' ) (ft,i, The situation is described by a two-step 
large deviations principle, 

¥ G [E G ,j = Ne]^e NhB{ - € \ P[L G ]xe M(L<3 '. (65) 

The rate functions are again computed through their Legendre transforms by considering two temperatures, yj and 
VG, 

F{y.j,yG)= Vaf - £(f,yj), Vg = d f K,(f,yj), (66) 
yjf(Vj)= yje-L(e), yj = 9 e L(e). (67) 



where the factor yj in front of / in the second line is introduced as in Eq. (|47|l to conform with the traditionnal 
definition of free energies. Taking Poissonian graphs as an example, we have explicitely the following expressions, to 
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be compared with Eqs. (|52() and l|53|) : 



k=0 



V l P °\ = y E^W / WVP.V^S \Po - P {k) m}]\ zW[{P i }]"°/"e-'xt k «\ 



1=1 



Po(h ) = pM[{Pi}}(h ) = ^Ej f HdhPiihWho - ti k >V*»({hi}))e-y j 

"* i=l 

Z = iW[{P i }]=E J f f[dh i P i (h i )e-^ Ai ^ ,W) ^\ 

^ i=l 

oo „ 2 / „ 2 



(68) 



Vg/vj 



-yjAE { / 12) (h u h 2 ) 



InZ 



In 



.fc=o 



£> 7 (fc) / HVPiPiPilslPo-P^iiPi}]] Z«[{P 4 }p/y.7 e -*x(fc,7) 



where Ej denotes the average over the couplings J. Note that with ya = 0, we obtain large deviations with respect to 
the couplings on a typical graph, while with yj = 0, we obtain large deviations with respect to the graphs for typical 
couplings. With ya = yj, the two sources of disorder are treated at a same level, in analogy with y = f3 in Sec. IIV Al 
and [i — y, (x = 1) in Sec. HVBl fthis prescription is sometimes referred to as the "Nishimori temperature" |39|). 

An interesting feature of the cavity equations is the possibility to implement them as a message-passing algo- 
rithm 36] to study for example here the large deviations with respect to the couplings on a given graph. The message 
passed along the oriented edge (i — > j) is the distribution p( l ~*J), which, in particular cases, can be parameterized by 
a finite set of real numbers, and the update rule is 



-y.,AE n ({hj}) 



(69) 



jei-o 



where the notation j 6 0-i means that, on the particular graph considered, j is a neighbor of different from i. This 
algorithmic scheme could be used to design graphs with optimal properties, with for instance applications in coding 
theory, in the context of low-density parity-check codes 



V. CONCLUSION 



While statistical physics of disordered systems have so far mostly focused on the thermodynamical properties of 
samples which are typical with respect to the source of quenched disorder, we have shown here that its methods 
can be extended to describe large deviations, that is, atypical samples. Large deviations are of foremost interest in 
probability theory and the approach followed here, though admittedly non rigorous, is based on clearly formulated 
assumptions which should be amenable to mathematical justifications. In its simplest form indeed, the LDCM we 
exposed assumes no replica symmetry breaking, a situation in which many of its typical predictions have been proved 
to be exact [il). 

From the perspective of algorithmic complexity, the LDCM can be seen as a first step in an attempt to reconcile the 
worst case and typical case analysis, usually regarded as antagonistic. However the scope of large deviations should 
not be regarded as restricted to optimization theory, as it notably allows to work out the statistical mechanics of 
physical systems with adaptive structures. An example of such system is constituted by random networks subject 
to mechanical constraints where the possibility to adapt leads to the occurrence of an intriguing intermediate phase, 
preceding a rigidity transition p| ; an other example along the same lines is given by proteins whose structure is shaped 
by strong constraints |42|. Adaptative structures in random graphs are also of interest in the seemingly unrelated 
field of neural networks |43l| . Finally, in the spirit of the most impressive achievements of its typical counterpart |36| , 
it would also be interesting to implement the LDCM on particular ensembles of instances to analyze, and possibly 
design, graph structures with specific properties. 
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